AugLy

last updated on 10/10/2023

Repository Documentation Transformations Paper

Supported languages:

Python (3.6+)

Supported ML libraries:

PyTorch

Requirements:

regex>=2021.4.4
python-magic>=0.4.22
iopath>=0.1.8
tqdm
dataclasses
portalocker
typing-extensions
importlib-resources
zipp>=3.1.0

AugLy includes various specific transformations that users perform in real life on Internet platforms such as Facebook, for example when they turn an image into a meme, overlay text/emoji on images/videos, or repost a screenshot from social media. Although AugLy also contains more generic transformations, it is particularly useful for working on issues such as copy detection, hate speech detection, or copyright infringement.

Transformations Description Tags

Blurs the image.

BLUR

Original	Augmented

`I.Blur(radius=1, p=1)`

Original	Augmented

`I.Blur(radius=5, p=1)`

Alters the brightness of the image.

PHOTOMETRY

Original	Augmented

`I.Brightness(factor=0.7, p=1.0)`

Original	Augmented

`I.Brightness(factor=1.3, p=1.0)`

Alters the aspect ratio of the image.

BASIC GEOMETRY

Original	Augmented

`I.ChangeAspectRatio(ratio=0.5, p=1.0)`

Original	Augmented

`I.ChangeAspectRatio(ratio=1.5, p=1.0)`

Scales the image up or down if necessary to fit in the given min and max resolution.

BASIC GEOMETRY

Original	Augmented

`I.ClipImageSize(min_resolution=None, max_resolution=20000, p=1)`

Adjust brightness, contrast, or saturation based on the provided parameters.

PHOTOMETRY

Original	Augmented

`I.ColorJitter(brightness_factor=0.5, contrast_factor=1.0, saturation_factor=1.5, p=1.0)`

Alters the contrast of the image.

PHOTOMETRY

Original	Augmented

`I.Contrast(factor=0.6, p=1.0)`

Original	Augmented

`I.Contrast(factor=1.2, p=1.0)`

Crops the image.

BASIC GEOMETRY

Original	Augmented

`I.Crop(x1=0.1, y1=0, x2=0.95, y2=0.8, p=1.0)`

Changes the JPEG encoding quality level.

WORSENING

Original	Augmented

`I.EncodingQuality(quality=0, p=1.0)`

Original	Augmented

`I.EncodingQuality(quality=30, p=1.0)`

Alters the image to be grayscale.

PHOTOMETRY

Original	Augmented

`I.Grayscale(mode='luminosity', p=1.0)`

Original	Augmented

`I.Grayscale(mode='average', p=1.0)`

10.

Horizontally flips the image.

BASIC GEOMETRY

Original	Augmented

`I.HFlip(p=1.0)`

11.

Applies given augmentation function to the masked area of the image.

COMPOSITION

Original Augmented

transform_function = I.Saturation(factor=0.4, p=1.0)

mask = Image.open('/home/sofia/Desktop/Projects/dalib/assets/images/dataset/031.png')

AUGMENTATIONS = [
     I.MaskedComposite(transform_function, mask, p=1.0)
]

12.

Creates a new image that looks like a meme, given text and an image.

COMPOSITION SOCIAL NETWORK

Original	Augmented

`I.MemeFormat(text='WHAAAAT?', font_file='/home/sofia/Desktop/augly_assets/fonts/OstrichSans-Black.ttf', opacity=1.0, text_color=(0, 0, 0), caption_height=250, meme_bg_color=(255, 255, 255), p=1.0)`

13.

Alters the opacity of the image.

PHOTOMETRY

Original	Augmented

`I.Opacity(level=0.3, p=1.0)`

Original	Augmented

`I.Opacity(level=0.7, p=1.0)`

14.

Overlay an emoji onto the original image.

COMPOSITION SOCIAL NETWORK

Original	Augmented

`I.OverlayEmoji(emoji_path='/home/sofia/Desktop/smile.png', opacity=0.8, emoji_size=0.15, x_pos=0.35, y_pos=0.20, p=1.0)`

15.

Overlays an image onto another image at position (width * x_pos, height * y_pos).

COMPOSITION

Original Augmented

overlay = '/home/sofia/Desktop/Projects/dalib/assets/images/dataset/026.jpg'

AUGMENTATIONS = [
     I.OverlayImage(overlay, opacity=0.7, overlay_size=1.0, x_pos=0.4, y_pos=0.4,
                    max_visible_opacity=0.75, p=1.0)
]

16.

Overlays the image onto a given background image at position (width * x_pos, height * y_pos).

COMPOSITION

Original Augmented

background_image = '/home/sofia/Desktop/Projects/dalib/assets/images/dataset/023.jpg'

AUGMENTATIONS = [
     I.OverlayOntoBackgroundImage(background_image, opacity=0.7, overlay_size=0.8,
                                  x_pos=0.4, y_pos=0.4, scale_bg=False, p=1.0)
]

17.

Overlays the image onto a screenshot template so it looks like it was screenshotted on Instagram.

COMPOSITION SOCIAL NETWORK

Original Augmented

template_filepath='/home/sofia/Desktop/augly_assets/screenshot_templates/web.png',
template_bboxes_filepath='/home/sofia/Desktop/augly_assets/screenshot_templates/bboxes.json',

I.OverlayOntoScreenshot(template_filepath, template_bboxes_filepath, max_image_size_pixels=None,
                        crop_src_to_fit=False, resize_src_to_match_template=True, p=1.0)

18.

Overlays stripe pattern onto the image (by default, stripes are horizontal).

COMPOSITION

Original	Augmented

`I.OverlayStripes(line_width=0.05, line_color=(255, 255, 255), line_angle=0, line_density=0.7, line_type='dotted', line_opacity=1.0, p=1.0)`

Original	Augmented

`I.OverlayStripes(line_width=0.05, line_color=(255, 255, 255), line_angle=45, line_density=0.5, line_type='dashed', line_opacity=1.0, p=1.0)`

19.

Overlays text onto the image (by default, text is randomly overlaid).

COMPOSITION SOCIAL NETWORK

Original Augmented

font_file='/home/sofia/Desktop/augly_assets/fonts/IBMPlexMono-BoldItalic.ttf'

I.OverlayText(text=[1, 2, 40, 41, 42, 43, 44], font_file, font_size=0.15,
              opacity=1.0, color=(255, 0, 0), x_pos=0.0, y_pos=0.5, p=1.0)

20.

Pads the image.

BASIC GEOMETRY

Original	Augmented

`I.Pad(w_factor=0.25, h_factor=0.10, color=(100, 50, 100), p=1.0)`

21.

Pads the shorter edge of the image such that it is now square-shaped.

BASIC GEOMETRY

Original	Augmented

`I.PadSquare(color=(0, 150, 200), p=1.0)`

22.

Applies a perspective transform to the image so it looks like it was taken as a photo from another device.

BASIC GEOMETRY

Original	Augmented

`I.PerspectiveTransform(sigma=50.0, dx=0.0, dy=0.0, seed=42, p=1.0)`

23.

Pixelizes the image.

TESSELLATION

Original	Augmented

`I.Pixelization(ratio=0.1, p=1.0)`

24.

Randomly changes the aspect ratio of the image.

BASIC GEOMETRY

Original	Augmented

`I.RandomAspectRatio(min_ratio=0.5, max_ratio=2.0, p=1.0)`

25.

Randomly blurs the image.

BLUR

Original	Augmented

`I.RandomBlur(min_radius=0.0, max_radius=10.0, p=1.0)`

26.

Randomly changes the brightness of the image.

PHOTOMETRY

Original	Augmented

`I.RandomBrightness(min_factor=0.0, max_factor=2.0, p=1.0)`

27.

Overlays a random emoji onto the image.

COMPOSITION

Original	Augmented

`I.RandomEmojiOverlay(emoji_directory='/home/sofia/Desktop/augly_assets/twemojis/smileys', opacity=1.0, emoji_size=0.15, x_pos=0.4, y_pos=0.8, seed=40, p=1.0)`

28.

Adds random noise to the image.

NOISE

Original	Augmented

`I.RandomNoise(mean=0.1, var=0.01, seed=42, p=1.0)`

29.

Randomly pixelizes the image.

TESSELLATION

Original	Augmented

`I.RandomPixelization(min_ratio=0.1, max_ratio=0.2, p=1.0)`

30.

Randomly rotates the image.

BASIC GEOMETRY

Original	Augmented

`I.RandomRotation(min_degrees=-15, max_degrees=-10, p=1.0)`

31.

Resizes the image.

BASIC GEOMETRY

Original	Augmented

`I.Resize(width=1410, height=1410, resample=2, p=1.0)`

32.

Rotates the image counterclockwise.

BASIC GEOMETRY

Original	Augmented

`I.Rotate(degrees=10.0, p=1.0)`

33.

Alters the saturation.

PHOTOMETRY

Original	Augmented

`I.Saturation(factor=0.6, p=1.0)`

Original	Augmented

`I.Saturation(factor=1.3, p=1.0)`

34.

Alters the resolution.

BASIC GEOMETRY

Original	Augmented

`I.Scale(factor=0.1, interpolation=None, p=1.0)`

35.

Alters the sharpness.

SHARPNESS

Original	Augmented

`I.Sharpen(factor=0.2, p=1.0)`

Original	Augmented

`I.Sharpen(factor=4.5, p=1.0)`

36.

Shuffles the pixels of the image with respect to the shuffling factor. The factor denotes percentage of pixels to be shuffled and randomly selected.

NOISE

Original	Augmented

`I.ShufflePixels(factor=0.5, seed=3, p=1.0)`

37.

Skews the image with respect to its x or y-axis.

BASIC GEOMETRY

Original	Augmented

`I.Skew(skew_factor=0.1, axis=0, p=1.0)`

Original	Augmented

`I.Skew(skew_factor=-0.3, axis=0, p=1.0)`

Original	Augmented

`I.Skew(skew_factor=0.25, axis=1, p=1.0)`

38.

Vertically flips the image.

BASIC GEOMETRY

Original	Augmented

`I.VFlip(p=1.0)`

Libraries

AugLy